In manufacturing sites or manufacturing plants in which a large number of manufacturing apparatuses are operated, the operation rates of the manufacturing apparatuses have a great impact on production volumes. In a case where a usually-avoidable problem resulting from neglect of regular maintenance and checkups or an unexpected problem has occurred in a manufacturing apparatus but the problem has not been handled efficiently, the manufacturing apparatus cannot be operated for a long time, which leads to decreases in the operation rate and the production volume. Therefore, it is desired that the operation suspension time of a manufacturing apparatus is reduced as much as possible. In performing maintenance, checkups and repairs, since the maintenance, checkups and repairs vary from manufacturing apparatus to manufacturing apparatus, there are some cases where an operator refers to an instruction manual or a checklist (hereinafter referred to as a checklist) showing a workflow of each work step.
Recently, wearable devices have been actively introduced into manufacturing sites. In such a manufacturing site, for example, operators wear eyeglasses-type wearable devices and refer to checklists displayed on the lens surfaces. In this way, the operators no longer need to refer to paper checklists while working on apparatuses, and consequently the operators can have their work done efficiently even in the case of unfamiliar and complicated work without interruption of the work.
However, the operators still need to check against checklists to ensure completion of their work at each work step. Therefore, paper checklist are still prepared even although the operators electronically refer to checklists on the screens while working on the apparatuses, and the operators stop their work to fill in the paper checklists at the end of each work step. Since the operations of manufacturing apparatuses are kept stopped during that time, the production volumes decrease. Further, after returning their office, the operators write reports on their work based on the checklists. It is quite cumbersome for the operators to write such reports on their work.
There is a system for supporting an operator by using a head-mounted display with a built-in camera. As an example of the system, there is a medical-equipment management system which supports an operator of a medical device such as a used and contaminated endoscope or a piece of medical equipment such as a scalpel or forceps.
The system includes a head-mounted camera for capturing an image of the sight of an operator of a medical device or a piece of medical equipment; storage means for storing an image of the sight of an operator captured when the operator demonstrates the medical device or the medical equipment as a reference image; first determination means for comparing the image captured by the camera and the reference image read from the storage means in order to determine whether a predetermined operation is performed normally based on similarity between these two images; data output means for outputting data indicating an alarm or an instruction based on a determination result of the determination means; and output means for outputting the alarm or the instruction to the operator based on the data indicating the alarm or the instruction.
The system automatically recognizes the operation of the medical equipment operator by comparing the operation of the operator captured by the camera with the reference image prepared in advance. However, in this method, since the recognition accuracy of simple pattern match between images is low, complicated image processing such as feature extraction from the images is further required. Consequently, as the automatic recognition processing becomes highly complicated, a considerable amount of time will be required for the image processing.
The present embodiment aims to provide a system and a method for recognizing movements of a user of a wearable device quickly, simply and accurately and displaying content to support the user based on a recognition result.